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The research project evaluates second grade economic 
materials developed over a three-year period with Des Hoines, Iowa, 
teachers enrolled in the Developmental Economic Education Program 
(DEEP) . The study measured the effectiveness of student teacher 
materials in the teachiug of basic economic understandings and an 
in-service program for teachers. The sample consisted of 24 teachers 
and 504 second graders. Three treatment groups included group one 
using the conventional social studies materials; group two using the 
specially designed economic education materials; and group three 
using the same materials as group two, but whose teachers had 
received in-service training. The five week study began with the 
administration of the SRA Test of General Ability (TOGA) and the 
Primary Test of Economic Understanding (PTEH) and ended with a second 
administration of the PTED as a post-test. Conclusions are as 
follows: 1) Basic economic concepts can be taught at early elementary 
grades and growth in students* understanding of them can bo measured; 
2) A close relationship exists between materials provided for 
teachers and students and pupil growth in economic understanding; 3) 
Full-scale programs in economic education for teachers may not fce 
necessary. Eleven recommendations are included in the report. 
(Author/PM) 
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I - INTRODUCTION 
Background of Present Study 

During the past decade, social studies programs produced for elementary 
schools have undergone extensive changes. New developments. Including Jerome 
Bruner*s emphasis on teaching the "structure of a discipline/' together with the 
Joint Council on Economic Education's Project DEEP (Developmental Economic 
Education Program) and associated "Project Social Studies," have had profound ef- 
fects upon the "new" programs now being published. 

The "revolution" began with the publication In 1963 of Senesh's Our Working 
World, which he describes as an "organic curriculum" for elementary social studies. 
His curriculum Is developmental and Inter-dlsclpllnary , containing basic concepts 
from not only economics, but from political science, sociology, anthropology, 
et al . Today, nearly all major publishers for elementary social studies materials 
are In the midst of publishing "new" social studies series. Such materials 
present elaborate scope and sequence charts to demonstrate the development of 
basic concepts in economics and other behavioral sciences as well as in history 
and geography in "inter" or "multi" disciplinary approaches to the social studies. 

Another source of materials has been developed by the authors in connection 
with th€! DEEP Project in the Des Moines, Iowa, Public Schools. The Des Moines 
School District was involved in Project DEEP as a "pilot" school system from 
1965-1968. The major thrust of Project DEEP in Des Moines has been in the area 
of economic education focusing on: (1) materials development, and (2) the in- 
service training of teachers, particularly at the elementary school level. 

Close liaison was maintained by the authors with Des Moines teachers in 
developing economic education materials for use In the primary grades in the 
Des Moines Public Schools. Such a close and continuous relationship was needed 
for at least two important reasons. First, once the economic concepts to be 
taught were determined, teachers provided invaluable insights into methods of 
"presenting them to children; and secondly, close contact with teachers pcovided 



needed Informaf Ion or thejf ovm ability to grasp or handle economic concepts. 
Although extrenely time-consuming, in-service programs involving teachers, cur- 
riculum specialists, and economists in the development of material probably are 
one of the best means of insuring that material developed will be presented to 
children in the intended manner. 

The first .naterial developed in the Des Moines DEEP Project was published 
by the Joint Council on Economic Education under the title GOODS, SERVICES AND 
PEOPLE in 1967. A further refinement of these materials resulted in the publica- 
tion of the book, THE CHILD'S WORLD OF CHOICES by The University of Iowa in 1968. 
THE CHILD'S WORLD OF CHOICES is essentially a teacher's manual and resource book 
for Grades K-3, providing a spiral development of economic understandings based 
upon five major economic generalizations. (See Appendix A.) 

Based on our experience of working with a considerable number of teachers 
over a protracted period of time, it seemed doubtful that the teacher who lacked 
formal training in economics could be expected to effectively teach economic 
concepts solely from a teacher ^s manual or resource book. Therefore, additional 
materials were developed which would more effectively meet the needs of such 
teachers. The materials, a Student Activity Book for The Child's World of 
Choicest Grade 2 , and a Teacher's Guide to the Student Activity Book for The 
Child's World of Choices were prepared and published in February, 1970, by The 
University ot Iowa. 

The Teacher's Guide to the Student Activity Book provides a general frame- 
work of activities and teaching strategics to be followed in introducing the 
children to the key economic ideas. The material is organized under the five 
generalizations, with appropriate student activities designed for each. Each 
lesson in the Teacher's Guide contains references to appropriate sections of 
THE CHILD'S WORLD OF CHOICES for content background, term definitions, and 
child-oriented activities as well as related ideas and understandings under 
-Bach generalization. As planned, THE CHILD'S WORLD OF CHOICES, the Student 
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Ac tivity Book , and the Teacher*s Guide to the Student Activity Book are to be 
regarded as a unified body of materials in 'che belief that their combined use will 
iasure the most effective teaching and learning processes. 

The Need for an Evaluation Instrument 

The new economic education materials developed for use in the elementary 
social studies curriculum, then, have become increasingly sophisticated. Empha- 
sis is placed on the development of children's understanding of basic concepts 
or fundamental ideas contained in the discipline of economics, with these concepts 
to be taught as part of the structure of the discipline. The new materials 
emphasizing the spiral development of concepts indicate an increased need foi 
greater coordination of teachers' efforts at different grade levels. 

Two basic assumptions have been predominant in the development of the nev' 
materials. First, the presumption has been that an examination of the ideas cr 
concepts contained in the materials will provide a fascinating and meaningful 
experience for children. It also has been assumed that teachers will underst^md 
the concepts and present them in an effective manner. However, since evaluation 
instruments rarely, if ever, accompany the materials, no concrete evidence exists 
to confirm or refute these beliefs. 

The almost revolutionary change in the character of the social studies 
indicates the overwhelming need for some evaluation instrument to determine 
children's ability to learn as well aj teachers' ability to teach the new TTiaterials. 
It may be that specially designed m?»terials are needed for children who differ 
sharply in their socio-economic backgrounds or in their intellectual capacities. 
In the past, capable teac^iers could be depended upon to make the necessary 
adjustments in the presentation of materials to compensate for differences in 
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students' backgrounds or abilities. However, unless many teachers are given 



special training, this may not be thu case for the new material. 

Related to this need for an evaluation instrument is the need for more 
information on the relationship between teachers' preparation in economics and 
the effectiveness with which new materials in economic education are presen'.ed 
in the classroom. If effective teaching of the new materials requires some 
knowledge of formal economic content, and if these materials are to be used in 
the classroom, then some way must be found to provide present teechers, as well 
as those preparing to teach, with the needed training. 

As was mentioned earlier, in recent years economics has received increased 

emphasis in the elementary grades. As is the case at the high school and 
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college levels, school systems need a recognized, acceptable instrument to 
evaluate economic understandings now being incorporated into the elementary 
curriculum over the nation. Preferably, evaluation instruments providing bench- 
marks and indicating students' economic understandings should be available for 
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use at the completion of both third and sixth grade. The following pages 



If the extent of exposure to economics of Des Melnes elementary teachers 
is typical, only a small proportion of such teachers are ever exposed to 
economics in either pre-service or in —service pi ograms . A study of Dcs Moines 
elementary teachers' transcripts revealed that over 70 per cent of the teachers 
had zero hours in economics. Even for those teachers who had received some 
formal preparation in eronomics, generally it had been ten or more years since 
their last courses were taken in this area. 

2 

Two major evaluation instruments measuring economic understanding have been 
developed for use in high F::hools and colleges. The first instrument, the Test 
of Economic Understanding , was developed by the Joint Council on Economic Educa- 
tion and published by Science Research Associates and has been widely usdd at 
the senior high school level. The second instrument, the Test of Understanding 
in College Economics (TUCE) , was developed by the Joint Council and published by 
the Psychological Corporation in 1958. This instrument has been used to evaluate 
the effectiveness of methods and materials used in the principles course in 
economics offered by colleges and universities. 

-^An evaluation instrument. Test of Elementary Economics (sixth grade level), 
has been developed by David C. Crosier and others at West Springfield, Mass., 
West Springfield Public Schools, 1969. The results obtained from using this 
instrument under experimental conditions may be obtained from Peter Sloane, 
Director, Center for Economic Education, Clark University, Worcester, Mass. 



describe the efforts of the authors in an initial attempt to develop a satis- 
factory testing instrument for use in the primary grades. 

Development of a Testing Instrument for the Primary Grades 

The development of a testing instrument for primary grade children proved 
a most difficult task, and several problems were encountered. First, the limited 
reading ability of primary grade children posed a special problem in choosing the 
test form. Secondly, the idea of developing a multiple-choice test consisting 
of pictures was tried but proved to be too time-consuming and costly an opera- 
tion for this research project. 

The authors then considered the use of a test with Yes-No items as well 
as an All-No test form. However, the Yes-No test form provides the subject 
with only two options and tends to have a low reliability. Test results also 
are difficult to interpret due to the acquiescence set of children. The All-No 
test has a higher reliability than the Yes-No test, but this test produces 
Spuriously high achievement scores for children who are not acquiescent. 

Of most value to us in selecting the test forms was a paper presented to 
the Education Section of the Utah Academy of Science, Arts, and Letters in 1968 
by A, Guy Larkins and James P, Shaver, then of Utah State University entitled, 
"Comparison of Yes-No, Matched Pairs, and All-No Scoring of A First-Grade 
Economics Achievement Test." 

Based on the Larkin-Shaver Study, the "Yes-No Matched Pair" format was 
adopted as the test form. This involves writing reversed items for each concept 
or bit of information tested, "Reversed" means that for every Yes-Item there is 
a No-Item intended to test the same content, and the "matched" items are scored 
as one. This technique has been devised to cope with the acquiescence-dissent 
biases and should cancel both effects. 

While the matched-pair technique increases the number of options on each 
item (item = matched-pair) from two to four, it also reduces the size of the 
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test by half. However, according to the studies performed by Larkins and Shaver, 
the positive effect on reliability of doubling the options on each item outweighs 
the negative effects of halving the length of the test, 

A serious problem was encountered in determining the number of items to 
be included in the test. If too small a number of test items are included, it 
is difficult to obtain a reliability level sufficient to justify the comparison 
of group means. If too large a number of test items are included, young children 
become excessively bored and fatiguefl. Administrative difficulties could be 
anticipated if a large test is subdivided and given to children in different 
periods • 

The decision was made that the test should consist of approximately 30 
matched-pair items or 60 individual items. It was thought that this test size 
would be adequate for reliability purposes and that it could be given in a 
thirty-minute period, which seemed appropriate to the maturity and attention 
span of primary grade pupils. 

Prior to the development of materials, a conceptual framework had been 
drawn up identifying the major economic understandings, major concepts, and 
sub-concepts which were to be developed under each of the five economic 
generalizations. The conceptual framework also was used to determine test items 
to be included in the evaluation instrument. 

Initially, a preliminary test was constructed, consisting of 69 matched- 
pair items (138 individual items) or more than double the number of items which 
were to be included in the final test. Due to the large number of items, this 
test was subdivided by economic generalization, and each part was administered 
on a weekly basis to the group of second grade children using the preliminary 
materials. 

The tests of students whose scores fell within the upper and lower 27 per 
cent categories were selected for item analy '.is , including determination of the 
Q level of difficulty and index of discrimination for each item. This information 
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v;as used to reduce the number of test items from 69 to 39 and to determine those 
items to be refined or revised. Comments on each of the items alro were solicited 
from other persons, including economists and educators familiar with the primary 
grades . 

The next step involved contacting a public school system which used mate- 
rial focusing on economic concepts in the first and second grades. This arrange- 
ment provided the authors with a group of children who could be considered to 
be knowledgeable in primary grade economics. The school agreed to administer 
the test to approximately 100 second grade children. Again, an item analysis 
was performed on the test results, liased on this information, and on the comments 
of several persons, the authors reduced the size of the test from 39 to 32 
matched-pairs, and additional items were revised. The 32 matched-pair items 
constituted the evaluation instrument which was later used in the Des Moines 
experiment . 

II - PURPOSES OF THE EXI^ERIMENT 

The specific purposes of this experimental study include: 

First, to measure the effectiveness of the specially designed student and 

teacher materials in the teaching of basic economic understandings at the second 

grade level. 

Second, to measure the effectiveness of an in-service program for teachers 
using the specially designed student and ceacher materials. 

Third, to measure the effectiveness of the economic education materials 
for second grade children in target and non-target schools. 

Fourth, to measure the effectiveness of the materia s for children who 
differ in characteristics such as age, sex, and levels of reasoning ability. 

Statement of Basic Hypotheses 
Or stated in another way, the experiment was designed to test the follow- 
>tng hypotheses : 
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1. As measured by the Primary Test of Economic Understanding (PTEU) , the 
mean change scores of the experimental groups who used special materials would 

be significantly different from the control group who did not use special materials. 

2. As measured by PTEU, the mean change scores of experimental groups in 
target schools would be significantly different from the mean change scores of 
experimental groups in non-target schools. 

3. As measured by PTEU, the mean change scores of experimental classes 
taught by teachers participating in the in-service program would be significantly 
different from the mean change scores of experimental classes taught by teachers 
not participating in the in-service program, 

4. There would be a significant relationship between pupils* performance 
on the PTEU and their age and sex. 

5. Pupils* scores on the PTEU would be more closely related to their 
scores on the TOGA reasoning test than on the TOGA information test. 

Ill - EXPERIMENTAL DESIGN 
Selection of Groups 

The experimental design involved 24 teachers in the Des Moines School 
System and their classes at the second grade level, in experimental and control 
groups, and involved approximately 500 pupils. As would be expc •-ed, a truly 
random sample is rarely possible in public school experimental situations. Thus, 
the "non-equivalent control group design*' described by Campbell and Stanley in 
Experimental and Quasi-Experimental Designs for Research in Teac hing and used by 
Sol Spsars in his dissertation, "An Evaluation of Three Economic Education 
Programs for First Grade*' was employed. In this design, experimental and control 
groups could be used but pupils could not be randomly assigned to these groups. 
Instead, intact classrooms could be randomly assigned to control and experimental 
groups . 
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Steps to Avoid Contamination of Sample 
Inasmuch as the materials to be used had been developed over a three-year 
period with Des Moines teachers enrolled in Project DEEP in-service classes, it 
was first necessary to eliminate all second grade teachers who had participated 
in Project DEEP, so that teachers in the experimental classes could approach 
the materials de novo . 

Secondly, in the random selection from the teachers and classes remaining, 
it was necessary to make certain that no two experimental classrooms were located 
in the same building. 

Thirdly, all testing, both "pre'' and "post/* was done by the principal or 
a consultant teacher with the regular teacher out of the room. At no time did 
the teachers in the experiment possess a copy of the tests used. 

Target and Non-Target School Classification 
A second consideration in setting up control and experimental groups was 
the matter of the "socio-economic level" of the pupils in the schools which 
participated in the experiment. The most convenient device available for 
making this distinction between schools was the designation of a "target" or 
"non-target" school as set forth in the guidelines of the U.S. Office of Education 
for determining a school's eligibility for financial aid under Title I of the 
Elementary and Secondary Education Act (E.S.E.A.). The designation of a "target- 
area school" means that certain criteria of socio-economic need and cultural 
deprivation qualify this school for special help under Title I funds. 

On the assumption that second grade pupils in target area schools would 
show less growth in economic understanding than pupils enrolled in non-target 
schools, an equal number of target and non-target classrooms were included in 
the sample for each of the three groups under investigation. 

At this point the categories to be used in the experiment might be 
described as follows: 



10 

CONTROL EXPERIMENTAL 





Cl 

Continue with regu- 
lar social studies 
program 


C2 

Use of THE CHILD'S 
WORLD OF CHOICES, 
including Teachers' 
Guide and Student 
Activity Book 


Use of THE CHILD'S 
WORLD OP CHOICES 
(as C2) plus in- 
service training 
sessions 


Target-Area 
Schools 


4 teachers and 
classes 


4 teachers and 
classes 


4 teachers and 
classes 


Non-Target 
Area Schools 


4 teachers and 
classes 

- 


4 teachers and 
classes 


4 teachers and 
classes 

- 



Experimental Treatment 
The "treatment," shown as C^^, C2, and C^ above, involved varying exposure 
of pupils to economic education as a part of classroom social studies instruc- 
tion during a five-week period of time as follows; 

1. Pupils in the eight classrooms included in the control group (C^) to 
continue with regular social studies program. 

2. Pupils in the 16 classrooms included in the experimental groups (C2, 
C3) to use special materials in economic education including THE CHILD'S WORLD 
OF CHOICES and related Teachers' Guide and Student Activity Book , 

In-Service Training Program 
The essential difference between the two experimental group treatments, 
C2 and C3, lies in the in-service training program for teachers in the C3 treat- 
ment group - 

While pupils in both C2 and C^ treatment groups were to receive instruction 
based upon THE CHILD'S WORLD OF CHOICES and the five basic economic generalizations 
developed :^n these materials, only the eight teachers in C^ received a weekly in- 
service training session during the course of the experiment. These teachers met 
from 3:30 to 5:30 each Monday afternoon for five weeks during the experiment to 

erJc 
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receive special instruction on each of the five basic generalizations to be 
taught during the experimental unit. Each teacher received $10 per training 
session from local Project DEEP funds. 

Measuring Instruments Used 
In view of the necessity of using Intact groups » the John C. Flanagan 
•'Test of General Ability" (TOGA), Form A, K-2, (SRA 1959), was administered to 
all pupils in both control and experimental groups so that suspected differences 
in ability levels could be determined and utilized in the analysis of outcome 
data. 

The TOGA was chosen because: (1) there were no other general intelligence 
test scores already available for the pupils involved and (2) TOGA is particu- 
larly suitable for testing second grade pupils since it is a test of general 
ability which does not require readings arithmetic, or any other form of school 
achievement • 

Although TOGA yields both an IQ score and a grade expectancy score, the 
wide range of ages found in the total sample of pupils went beyond established 
IQ and grade expectancy norms for the K-2 level of the test used. For the purposes 
of this experiment a raw score to IQ transformation would servp no purpose but 
would introduce error. Consequently, total TOGA raw scores were used as ability 
measures , 

Since TOGA also yields part scores for: (1) the child's Information about, 
or familiarity with, the world around him, and (2) the child's powers of abstract 
reasoning, these part scores were used to "test out" a tentative sub-hypothesis 
"that pupils' scores on the PTEU would be more closely correlated with pupils' 
scores on the TOGA reasoning test than on the TOGA information test." 

The second major measuring instrument used was the Primary Test of Economic 
Understanding (PTEU) discussed above. It contains 32 yes-no, matched-pair items 
which consist of short, declarative statements, each calling for a "yes" or "no" 
response by the pupil. Further, for every item in which the correct response is 
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"yes/* there is a matching item for which the correct response is "no/* Both 
matched items must be correct to count toward the pupil's score (See Appendix 
B for infotiiuition on difficulty and discrimination indices,) 

The use of these measuring instruments in the experimental process was as^ 
shown below: 



Groups 


Pre-Test 


Treatment 


Post-Test 


Cnnt ml ( C 1 

4 Target 

4 Non ^Target 


TOGA 
& 

PTEUj^ 


Regula*. social studies curriculum 
(5 weeks) 


PTEUj 


Experimental (C^) 


TOGA 
& 

PTEUj^ 


Use ol^ THE CHILD'S WORLD OF CHOICES 
materials — no-in-service training 
(5 weeks) 


PTEU2 


A Target 

4 Non-Target 


Experimental (C^) 


TOGA 
& 

PTEUj^ 


Use of THE CHILD'S WORLD OF CHOICES 
materials plus in-service training 
for teachers (5 weeks) 


PTEU2 


4 Target 

4 Non -Target 



To reiterate, in every case, the test materials (both TOGA and PTEU) were 
in the hands of building principals and were administered by them or consultant 
teachers only. (See Appendix C for list of schools, teachers, and principals 
involved in the evaluation experiment.) 

Project Calendar 
Week of March 30-April 3, 1970 

1. Orientation meeting held for all teachers and principals involved in 
the evaluation projects TOGA and PTEU tests distributed to principals. THE 
CHILD'S WORLD OF CHOICES materials distributed to teachers of all experimental 
classes. 



The reliability used in this study was derived by use of the Kuder-Richardson 
Formula 20, A reliability coefficient of ,79 was obtained for the test. The 
standard error of measurement was 2,47, 



2. First in-service meeting for the eight teachers in treatment held. 
To meet each Monday for five weeks of the project. 

3. TOGA and PTEU pre-test a( .ministered to all groups, control and experi- 
mental^ during this week, 

4. Teaching of economic understandings from THE CHILD'S WORLD OF CHOICES 
materials began in experimental groups C2 and C^ — to be taught at least 30 minutes 
daily, four days per week, as social studies unit. 

May 4 -May 6, 1970 

1. PTEU post--test administered to all groups, control and experimental. 
End of project activities in the classroom. 

2. Return of all test materials to project director at School Board office. 
It should be pointed out that each teacher and principal involved had been 

personally contacted by the project director and the purposes and outlines of 
the project explained before the orientation meeting, mentioned above, was held. 
Inclusion in the project was not determined by admini3tratiV3 fiat; each teacher 
and principal contacted had the option of withholding participation of any 
class or school. However, there were no refusals to participate. In fact, 
eagerness to participate was the rule. 

IV - ANALYSIS OF OBTAINED DATA 
Schematic Organization of Data 
Before consideration of the statistical analysis of data gathered from the 
experiment, an examination of the schematic organization of data classification 
as shown in Figure 1 below would be helpful. 

FIGURE 1 
^1 

Target (1) 11 
Non-Target (2) 21 



^2 
12 



22 



3 
13 



23 



In Figure 1, the first digit within each cell is the school classification 
(1 - Target, 2 » Non-Target). The second digit in each cell is treatment 
(C^ » Control, C2 « Materials, « Inservice and Materials). 

Within each cell in Figure 1 there are four classrooms The classroon^ is 
the sampling unit in the analysis. Thus each classroom contributes one score on 
each of the two variables, TOGA and PTEU^^. In this case that is the mean for 
the students in that classroom. Each cell, then, contains four such scores 
for each test. In addition, each cell has a mean (the arithmetical average of 
the means for the four classrooms). When the scores for all students within a 
cell are averaged (as opposed to averaging the means of the four classrooms) , the 
weighted mean for the cell is obtained. 

Classrooms-Wi thin-Cell Analyses 

The total number of students involved in the experiment and for whom scores 
on all measures (TOGA, PTEU^^, and PTEU2) were obtained was 504. Because intact 
classrooms were available for the experiment, classrooms-within-cell analyses 
of the data were planned. 

The first step in analyzing the data was to test for suspected differences 
among classrooms. In some situations, the classrooms-within-cells design might 
be used initially because differences among classrooms are suspected but not 
known tc exist. An F-test might then be made. If this F-test proved nonsignif- 
icant » one might then, on the assumption of no classroom differences, regard 

5 

this as a simple randomized design. 



•^Lindquist, E. P., Design and /malysis of Experiments in Psychology and 
O jducation , Houghton-Mifflin Company, Boston, 1953, p. 173. 
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The F-test for significance of the classroom differences was made for TOGA 
and PTEUj^ for each cell.^ In the present teminology , the F could be stated 
as the mean square for classrooms within a cell divided by the mean square for 
students within those classrooms. The results are summarized in Table 1. 

It will be noted that, in all cases except the target-treatment 2 group 
(cell 12), the F was significant for TOGA or PTEU^. Thus, it could be assumed 
that the assignment of students to classtooms had not been done randomly and 
that such classroom differences dictated that a classrooms-within-cells analyses 
be employed. As Lindquist has suggested, the classroom and not the individual 
child was treated as the unit of sampling. This Indicates that the cell did not 
contain a random sample of students, but more appropriately a random sample of 
classrooms. 



The use of the F-test in this situation is relatively uncommon, and 
therefore it may be beneficial to describe the procedure. Consider an experiiDent 
similar to the present one, where 504 students are available for a design requiring 
6 cells. Ideally, one would randomly assign 84 students and 4 teachers to each 
cell. Each teacher would then be randomly assigned 21 students, and there 
should be no significant differences among the 24 classrooms. 

If, in the present case, the requirement for target and non-target schools 
would prohibit such a procedure-- because the 504 students are already divided 
into target and non-target on the basis of their school's status — the ideal 
procedure would be random assignment of students to the classes within the 
cells in the target classification or to the classes within the non-target 
classification. If this were done, there should be no significant differences 
among the 12 classrooms within each classification. 

In actual practice, random assignment is seldom possible. In the Des Moines 
experiment, 12 classrooms were available to be randomly assigned to cells within 
each classification. The question to be asked is "Does the makeup of the classes 
in the cells differ from what it would have been had students been randomly 
assigned to cells and then randomly assigned to teachers?** The F-test provides 
an Index of this difference . 

A large value of the F indicates that the classes are aore different from 
one another than one would expect from randomly assigning students to classes. 
This is usually the result, because the students within a class come from the 
same school, neighborhood, etc. A student has more experiences in common with 
his classmates than with the other children who are assigned to the same cell. 

^Lindquist, pp. 173-174. 
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Table 1 

Summary for Class rooms-Wi thin-Cell Analyses 





Degrees of 


F-values 




Cell number 


freedom 


for TOGA 


PTEU 


11 


3,60 


.06 


3.16*** 


12 


3,65 


.90 


.72 


13 


3,68 


10.63*** 


.95 


21 


3,97 


7.60*** 


4.83*** 


22 


3,86 


2.56** 


1.79* 


23 


3,104 


1.31 


9.04*** 



***Signlficant Q .05 level. 
**Signlflcant @ .10 level. 
^Significant Q .20 level. 

Analysis of Covariance 

Since significant differences in both the TOGA and the PTEU^ scores existed 
among classrooms, the design chosen was an analysis of covariance. This analysis 
employed the TOGA scores as the covariate and used the differences between the 
PTEU2 and PTEUtl score for each class as the criterion score. This resulted in 
an analysis of 24 classrooms, treating each classroom as an individual case. 

Thus, for each classroom, the mean score on TOCtA was used as the covariate 
and the mean change score (PTEU2 minus PTEUj^) was used as the criterion variable. 
This amounts to analyzing the actual change in mean scores for classes in each 
treatment, adjusting these change scores to account for Initial differences in 
ability as measured by the TOGA. Stated another way, this adjusts final scores 
for initial differences on both PTEU-j^ and TOGA. 

A two-way (or two-factor) analysis was performed. One factor was treatments, 
with the three treatments being: control, materials, and materials plus in- 
service. The second factor was school classification: target or non-target. 
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The results of the two-factor analysis of covariance Is given in Table 2. 
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Table 2 



Summary of Two-Way Analysis of Covariance 



DF 



Mean square 



F 



P less than 



Within cells 

Treatments 

Target non-target 



17 
2 
1 
2 
1 



1.858 
11.854 
0.728 
1.046 
9.756 



6.379 
0.392 
0.563 
5.250 



.009 
.540 
.580 
.035 



Intersection 
Regression 



Check for Homogeneity of Regression 



The data in Table 2 indicate that the treatment effects were significant 
beyond the .01 level. Neither the interaction nor the classification effect 
was significant which means that the amount of change and the effectiveness of 
the treatments in economics achievement did not differ in the target and non- 
target classifications. This is true even though, as shown later in Table 6, 
there were significant differences in the scores (not adjusted change scores) 
of the target and non-target students. 

The findings of no significant interaction and no significant difference 
in the adjusted change scores for the target and non-targec classifications 
prompted an analysis of treatment differences for the combined classifications. 
Thus, within each treatment, the target and non-target classrooms were combined. 



^It is always important to check on the condition of homogeneity of regres* 
sion before interpreting an analysis of covariance. In tvie present case, the 
appropriate F-statistic for homogeneity of regression, with 5 and 12 degrees 
of freedom, MS among classroom regression/MS deviation of classrooms from cell 
regression, was equal to .32 which indicates that the condition vas well 
satisfied. (The teat for the significance of an obtained F-value is based on 
determining how unusual such a large value of F is; that is, the degree to 
which the mean square ratio exceeds 1. Wtten the value is less than 1, the 
condition is better satisfied than one would expect from random sampling.) 

In this instance, the condition referred to is the condition of homogeneity 
of regression of the change scores on TOGA total scores over the six groups. 
Had one randomly assigned 4 mean change scores to each cell, he would not expect 
to get the condition to homogeneity of regression as well satisfied as is the 
case here . Thus the interpretation of the analysis of covariance is justified. 
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Table 3 
Means of Treatment Groups 

Control (C],) (C2) (C^) Total 

Means on TOGA (X) 50.3820 A6.3054 49.1090 48.5988 

Means of Change Scores 

(PTEU2 " ^"^^i^ -^^^^ 2.1175 2.4055 1.7008 

Means of Adjusted Change Scores 

(Y - .2027 (X - X)) .2180 2.5824 2.3021 1.7008 

Table 3 presents the means of the treatment groups used in this analysis. 
The adjusted mean change score for each group was computed by the formula 
Y - b (X X) , where 

Y is the irean change score for that group; 

X is the mean TOGA score for that group; 

X is the overall mean TOGA score; and 

b is the mean regression coefficient, a within-group beta. 
For example, the computation of the mean adjusted change score for treatment 
group 1 is as follows: 

.5795 - .2027 (50.3820 - A8.5988) « .2180 

These adjusted change scores can be interpreted as estimates of the actual 
amount of change that would have resulted if the treatment groups had equal 
TOGA scores. These adjusted change score means for the three treatments were 
compared via t-tests. The results of the t-tests are presented in Table 4. 

From the analysis in Table 4, it is indicated that both experimental 
treatments were signficantly more effective than was the control method, and 
there is no indication of superiority for either experimental method . 

Relationships Among Variables 
Table 5 is presented to indicate the re?vationships found among the variables. 
O >uch a table is helpful in understanding what is measured by the economic tests, 
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Table 4 
Significance of Treatments 

Comparison T value (df « 20)* Significance 
with C2 3.35 .01 
with 3.14 .01 
with -.41 .70 

^Adjusted mean square used as the common error term. 
This error term contains the variance from three sources 
In Table 2: within cells, interaction, and regression. 

Table 5 

Correlations Among the Variables 
(n - 504) 







TOGA 


TOGA 


TOGA 








Age 


Info. 


Reas. 


Total 


PTEU^ 


PTEUj 


Sex 


.09 


.15 


.08 


.03 


-.01 


.01 


Age 




-.06 


-.07 


-.08 


-.06 


-.01 


TOGA Information 






.50 


.85 


.42 


.53 


TOGA Reasoning 








.88 


.36 


.45 


TOGA Total 










.45 


.57 


Pre-test (PTEUj^) 












.69 



Post-test (PTEUj) 

PTEUj and PTEU2. One interesting finding reported in this table concerns the 
correlations between the economics test and the part scores on the TOGA. The 
correlation between TOGA-lnf ormatlon and PTEU2 is .53, while the correlation 
between TOGA-reasoning and the PTEU2 is .45. These correlations are (statis- 
tically) significantly different, which means that PTEU is more closely related 
to information (as measured by TOGA) than reasoning. 

The data contained in Table 5 also indicate a very low correlation between 
children's scores on the economic achievement test and children's age and sex. 
While the correlation between PTEU scores and age was statistically significant, 
this relationship is of no practical significance. The child's sex is of even less 
-i mportance for explaining his or her achievement on the economics test. 
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In order to permit the reader to better determine the generallzablllty 
of the findings, data are presented in Table 6 describing the performance of 
the students who participated in the present study. Table 7 presents unweighted 
group means on relevant variables. 

Table 6 

Means (with standard deviations in parentheses) 
of Pre-test Scores (PTEU^^) for All Students 

^1 ^2 ^3 

Target 10.0 (4.2) 9.8 (3.9) 9.3 (3.9) 

Non-target 15.8 (4.7) 11.2 (4,0) 13.2 (4.7) 

Means (with standard deviations in parentheses) 
of Post-test Scores (PTEUj) for All Students 

Cl C2 S 

Target 10.5 (4.8) 10.9 (5.1) 11.7 (4.9) 

Non-target 16.3 (4.3) 14.4 (4.6) 16.3 (4.9) 
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Table 7 

Mean* TOGA Scores for All Classes 



Target 
Non-target 





^2 




47.4 


42.1 


45.9 


53.4 


50.5 


52.3 



Mean* Unadjusted Change Scores (PTEU2 minus PTEU^^) 



^1 S ^3 

Target J3S» 1.1 S^m 

Non-target 3.2 



Mean* Change Scores, Adjusted on TOGA 







S 


0.8 


2.7 


2.4 


-0.6 


2.7 


2.1 



Target 
Non-target 
Treatment 

means -f .1 2.7 2.25 

Cj^ Is control 
C2 is materials 

C^ is materials and in-service training 
* is unweighted average of class means 



V - SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 

Summary 

Tne Problem 

1. To measure the effectiveness, over a five-^week period, of specially 
designed teacher and student materials to be used in the teaching of selected 
economic concepts to children at the second grade level. 

2* To measure the effectiveness of an in-service program for teechers using 
the specially designed student and teacher materials. 

3. To measure the effectiveness of the economic education materials for 
second grade children in non-target as well as target schools. 

4. To measure the effectiveness of the materials for children who differ 
in such characteristics as age, sex, and levels of reasoning ability. 

The Procedure 

To evaluate the effectiveness of the materials, three treatment groups — 
^1» ^2* ^3 — ^^^^ used. Group C-^ used the conventional social studies materials 
already in the classrooms. Group C2 used the specially designed economic educa- 
tion materials for students and teachers. Group C^ was furnished the same 
materials as Group C2, but teachers in C^ were also involved in a five-week, 
or ten-hour, in-service program. 

The oc.mple selected for the study was drawn from existing second grade 
classrooms from the sixty elementary schools in the Des Moines school district 
except that teachers who had formerly participated in Project DEE'P in-service 
programs were initially excluded from the sample. Twenty-four classrooms and 
teachers were included in the final sample, and complete data were compiled on 
a total of 504 pupils. 

Of the twenty-four classrooms, eight were included in each of the three 
treatment groups. Further, to obtain data relevant to effectiveness of the 
materials for children of differing* socio-economic backgrounds, an equal number 
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of target and noi^arget classrooms were Included in the sample for each of the 
three groups under investigation. 

The study was conducted for a period of five wee|(S beginning with the 
administration of the SRA Test of General Ability (TOGA) and the Primary Test 
of Economic Understanding (PTEU^) and ending with a second administration of the 
PTEU as a post-test (PTEU2) . 

Findings and Conclusions 

On the basis of the data analyzed in this research, the following findings 
were obtained: ^ 

1 . The mean change scores of the experimental groups using the specially 
designed student and teacher materials were statistically significant and higher 
than the control ^roup who did not use the special materials . 

The first hypothesis of this research predicted that this difference 
would result; that both the experimental treatments would prove more effective 
than would the control method. This hypothesis was upheld by the data; the dif- 
ference between experimental and control group mean change scores on the PTEU, 
after adjustment for ability differences as measured by TOGA, proved statistically 
significant a<" .01 level. 

2. There are statistically significant differences in the scores on 
PTEUj^ and PTEU^ between target and non-target classifications . 

Pupils in target schools scored significantly lower on both PTEU pre- and 
post-tests than pupils from non-target schools in this study. However, statistical 
analysis of mean change scores (PTEU^ minus PTEUj^) , when adjusted for 
differences in pupil ability as measured by TOGA, showed no statistically 
significant difference between the target and non-target classification. That 
is to say, when change scores were adjusted to assume that both treatment 
groups had equal TOGA scores, the amount of change between the administration 
^of PTEU^ and PTEU^ was no longer statistically significant between target and 
E RJ C pn - 1 ar ge t experimental groups. 
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The data clearly shows that pupils from non-target areas scored higher on 
both the Test of General Ability and vhe Primary Test of Economic Understanding 
and that these same pupils achieved higher change scores as measured by the 
PTEU post-test. However, pupils from the target area classes also showed 
improvement in economic understanding as measured by the PTEU post-test. Thus, 
the use of the special materials in both target and no^-^arget classes proved 
effective; but any valid measurement of the extent of that difference must first 
take into account the significant degree to which target pupils differ from 
non-target pupils in general ability as measured by TOGA. 

3 . No statistically significant differences were found in the adjusted 
mean change scores for experimental classes taught by teachers involved in the 
in-service programs as compared with scores obtained by experimental classes 
of teachers not involved in the in-service program . 

There is no indication of superiority for either experimental method; 
however, the adjusted mean change scores obtained by both experimental treatment 
classifications proved to be statistically significant when compared to results 
obtained by the control groups. This would indicate that the specially prepared 
materials for teachers and students were a major determining factor in the marked 
differences in achievement of experimental groups over the control groups. 

The original hypothesis that the in-service program would make a 
statistically significant difference was not supported by the obtained data. 

4* A very low correlation was found between children's scores on the economic 
achievement test and the children's age or sex . 

While hypothesis 4 of this research predicted significant differences 
resulting from these two variables, they were found to be of such low correla- 
tion with economic understanding as to be of li::tle or no practical significance 
in terms of their prediction value concerning the use of the special materials 
or acquisition of economic understandings. 

ERLC 
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5. Pupils* scores on the Primary Test of Ecnomic Understanding were found 
to be more closely related to the TOGA part score on information than to the 
TOGA part score on reasonin|^ . 

Hypothesis 5, which predicted that there would be a higher correlation 
between PTEU and the TOGA part score on reasoning, was not supported by the data. 

While a student's score on PTEU pre-test appears to be the best predictor 
of what he will achieve on PTEU post-test (correlation coefficient .69), it 
should be noted that the correlation coefficient of .57 between PTEU^ and TOGA 
total as shown in Table 5 (page 19) would indicate that PTEU and TOG., do not 
just measure the general ability level of the student. By squaring the correla- 
tion coefficient .57, one finds that only about 32 per cent of a student's 
score on PTEU2 can be attributed to his TOGA score. The remainder of the score 
apparently depends mainly upon his understanding of econoniic content or is 
attributable to measurement error. 

Conclusions 

1. Basic economic concepts can be taught at early elementary grades and 
growth in students' understanding of them can be measured « 

2. A close relationship exists between materials provided for teachera and 
students andpupil growth in economic understanding. 

3. To the extent that elementary teachers are provided with sufficient 
background on economic content in teachers' materials, the necessity for full- 
scale programs in economic education for teachers may not be necessary. 

Recommendations 

1. This particular research des^.gn should be replicated at the same grade 
level to determine the universality of the Des Moines findings. 

2. This particular design could be replicated by using the same materials 
at the third grade level to measure any possible effects of greater etudent matu- 

j^l^^Tity in acquiring economic understanding. 
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3. To provide additional information needed for material and test revision, 
this particular research design should be replicated at the third grade level in 
school systems where presently the social studies program for grades 1-3 already 
incorporates economic understandings. 

4. Teacher and student materials should be developed for grades 1 and 3 
to ensure a planned sequential development of the five economic generalizations 
from THE CHILD'S WORLD OF CHOICES. 

5. Further data on the test should be obtained from upper level students 
In grades 4-6 where programs contain planned lessons in economic understandings. 

6. Since the materials are designed to be used in a spiral fashion over 
the primary grades, research designs are needed to measure the cumulative effects 
of spiral treatment of these five economic generalizations. 

7. While this study revealed no measurable advantage of in-service sessions 
for teachers, indices of discrimination arrived at through item analysis reveal 
that several test questions pertaining to central or basic economic concepts 
were missed almost universally. This would indicate that those concepts were not 
developed properly in the student-teacher materials, or through teacher presenta- 
tion, or were beyond the maturity level or developmental stage of most pupils. 
Before a revision of either the test or materials is undertaken, it vould seem 
desirable that these concepts might be more fully developed as focal points in 

an in-service program for teachers. 

8. More research should be done in the area of teaching economic concepts 
to socio-economically disadvantaged students. This study showed that such 
students could achieve growth in economic understanding through the use of 
special material. But ikiformation is needed on the degree to which teachers 
have to make adaptation in materials and teaching strategies in order to 
achieve greater economic understanding on the part of such pupils. 

9. Similai research should be undertaken in another social science 

O „discipline to see if the same general results are obtained in terms of target and 
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non-target groups and in terms of control and experimental groups. Evaluation 
instruinents are needed to measure pupil growth In cognitive aspects in other 
behavioral sciences as well as in economics. 

10. A study is needed of the relationship between the amount of time 
students are directly engaged in studying economics and their achievement in 
economic understanding. 

11. A study should be made of the effects of teacher familiarity with 
special materials. This study could be replicated using the same teachers but 
using next year's class of students. This would provide a measure of the effec^ 
of teachers* familiarity with materials on student achievement in economic 
understanding. 
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Appendix A 

Developmental Framework for Five Economic Generalizations 

ECONOMIC GENERALIZATION I; Because of linri-ted income^ consuming unite must 
choose which of their many wants for goods and services they will 
satisfy through purchases in the marketplaoe . 

Major Economic Understandings 
1« Households have many wants for goods and services. 

2* Most household wants for goods and services are satisfied through purchases 
in the marketplace* 

3. Hoxisehold purchases of goods and services in the marketplace require the 
use of money income. 

4. Household money income Is limited, relative to the kinds and amounts of 
goods and services desired. 

5. Households must decide what goods and services will be purchased with 
their limited money Income. 

6. Decisions by households to purchase certain kinds of goods and services 
mean that they must forego other kinds of goods and services. 

Major Concepts 

Scarci ty 

wants 

income 
Choice making 

income 

tas tes 

prices 
Opportunity cost 

Sub Concepts 

Consuming unit 
Consumer 
Consumer good 
Consumer services 
Economic goods 
Free goods 
Business unit 
Marketplace 
Money income 
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ECONOMIC GENERALIZATION II: Scarce resourcee are required for the produc- 
tion of goods and eervicee. 

Major Economic Understandings 

1 . There are not enough rebources to produce all the goodr and services 
people want. Consumption is limited by the scarcity of resources. 

a. Production inust precede consumption. 

b. Production requires the use of scarce re^^ources. 

2. Resources are versatile. 

a. A resource may be used in the production of many different goods 
or services. 

b. hfost goods and services may be produced with different combinaticms 
of resources. 

3. The use of resources involves choice making. 

a. Choices must be made between the various goods and services which 
could be produced. 

b. Choices must be made between the various combinations of resources 
which could be used in the production of a good or service. 

4. The use of a resource In the production of one good or service precludes 
its use in the production of others. 



Major Concepts 

Scaicity 
Vereatility 
Choice making 
Opportunity cost 



Sub Concepts 

Production 

Reeourcee - - land^ labor ^ capita I 

Bueineee firm — it ie the reeponeibility of 
persone within bueineee firms to make 
decisions concerning the use of scarce 
resources 
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ECONOMIC GENERALIZATION 111: Households earn money income by eelling the 
eervioea of their productive resources to businesses and, in turn, use 
household income to purchase goods and services from businesses. 

Major Economic Understandings 

1. Households earn money income through the sale of the services of their 
productive resources to businesses. 

a. Private ownership of productive resources. 

b. Production mainly carried on within business organizations rather 
than within households. 

2. The aaount of income earned by households mainly depends upon the aaount 
ar>d kinds of resources sold to businesses. 

3. The ancunt of consumer goods purchased by households from buslneaaea 
mainly depends upon the anount of income earned by houaeholds froiD the 
sale cf their resources to busiresses. 

4. Money Is constantly flowing from households to businesaes and from 
businesses to households as households buy goods from businesses 
(product market) and sell the services of their productive reaourcea 
to businesses (factor market). 



Major Concepts 

Marketpi^ace 

product 

factor 
Factor ownership 

private property 
Business firm 

buyer of services of productive factors 

producer of consumer goods and services 

seller of consumer goods and services 
Production — combining of factors (see Economic Generalization II) 
Household 

buyers of consumer goods (spending of income) 
sellers of services of productive resources (income earning) 
Money income 



Sub Concepts 

Barter 
Factorc 

Factor payment (expenditures^ price ^ income) 
Money - medium of exchange 
- standard of valve 

Prices 
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ECONOMIC GENERALIZATION IV: Some of f>€ople's Donts for goods ar.d e^rvioee 

are eatiefied through govemment. 



Major Economic Understandings 

1. Covemmcnt, as well as businesses, provides goods and services to 
households . 



a. Government provides some goods and services which would not be 
provided by private businesses. 

b. Government provides goods and services which individuals can 
purchase in the marketplace, but which many households would not 
purchase in the quantities which society considers adequate. 



2. Goods and services provided through goveimment require the use of scarce 
resources in their production. 



a. Some goods and services which government provides are produced 
directly by government, using resources which government must 
purchase . 

b. Some goods and services which government provides are produced by 
private businesses and purchased by government. 



3. Government requires money income to purchase resources and goods and 
services . 



a. The major source of government's money income is taxes. 

b. Taxes permit government to provide some goods and services to 
households, but leaves less money income in households for direct 
purchases from businesses. 

4. The provision of government goods and services involves choice iraking. 

a. Choices must be made between thr. various public and private goods 
and services which could be protluced. 

b. Choices must be made between the various public goods and services 
which could be produced. 

5. The opportunity cost of the use of resources in the production of 
government goods and services Is the other goods and services* both 
public and private, which could have been produced. 



Major Concepts 

Government provision of goods and cervices 
Government in the marketplace 
Government's need for money income 
Scarcity 
Choice making 
Opportunity cost 



Sub Concepts 
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Public goods and services 
Taxes 



ECONOMIC GENERALIZATION V: HauB^holdB -xn/ bom part of thtvT monmy inoom. 



Major Economic Understandings 

1. Saving Is another alternative use of houaehold incone. 

2. Houaehold saving la that part of houaehold Incoma which renalna after 
the payoent of taxes and the purchaae of conauwr goods. 

3. Vhen households save part of their Incow this nsans that they aust 
presently forego some of the goods and services vhich they, othervlae» 
could have purchased. 

A. When households save part of their current incoM this aaana that 
their purchaaes can exceed their houaehold incone (in the future). 



Major Concepts 

Dieposition of household 

foMSj oonBumption, Boving 

HouB0hold eaving 
S " HH - (Tx C) 

Choice making 

C VB, S 

PvBBent ue. future oonBumptian 
Opportunity cost of BWing 
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Appendix B 

Difficulty and Discrimination Indices for 
Final Test, PTEU^ 



Item 


Index of 
difficulty 


Index of 
discrimination 


Item 


Index of 
difficulty 


lodex of 
discrlmlnatioa 


1-33 


68 


.53 


17-49 


39 


.56 


2-34 


41 


.43 


18-50 


33 


.31 


3-35 


56 


.46 


19-51 


67 


.62 


4-36 


46 


.58 


20-52 


35 


.60 


5-37 


12 


.01 


21-53 


23 


.15 


6-38 


69 


.34 


22-54 


49 


.68 


7-39 


43 


.44 


23-55 


34 


.45 


8-40 


34 


.48 


24-56 


39 


.37 


9-41 


42 


.70 


25-57 


51 


.52 


10-42 


48 


.56 


26-58 


a 


.08 


11-43 


35 


.40 


27-59 


31 


.30 


12-44 


54 


.38 


28-60 


25 


.34 


13-45 


44 


.36 


29-61 


14 


.13 


14-46 


62 


.59 


30-62 


51 


.48 


15-47 


88 


.31 


31-63 


67 


.53 


16-48 


52 


.50 


32-64 


17 


.03 



The data include test scores for students falling in the upper and lower 
27 per cent of the total group, consisting of the 16 experimental classes. 
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Appendix C 

The follovlng information identifies the schools » teachers , and principals 
participating in the Des Moines evaluation study 



School 


Teacher 


Prinrinal 


Byron Rice 


Dana Gay lord 


Patience Guthrie 


Br voks 


Rosamond Ramsey 


Donald Williams 


T>ouglas 


Lucille Peterson 


Marjorie E. Schwien 


Dunlap 


Mary Shaffer 


Dale Jacobus 


Findley 


Edith Hand 


Nadine Machesney 


Carton 


Marjorie Sergeant 


Lorraine McFadden 


Grant 


Sara Marguardt 


Bernard Miller 


Hillis 


Virginia Smith 


Eleanor Singer 


Hoyt 


Marcia Pollock 


Ruth Collins 


Losan 

o 


Aneelyn Kins 


Donald D. SheK 


Lucas 


Peeiry Good 


James Mitchell 




Vivian Lucas 




Mann 


Bemiece Wright 


Violet Coldren 


Hadison 


June Malliet 


Kathryn B. Christian 


McKee 


Elizabeth Ruess 


James Daugherty 




JoAnn Vestal 




McKinley 


Blanche Rinehart 


Snowden Moon 


Hon roe 


Roberta Waldo 


Loretta Patrick 


Moult on 


Marsha Clark 


Cecil Leonard 




Jeraldine Kubicek 




Park Avenue 


Virginia Armentaro 


Irene Perkins 


Scott 


Marchie Glllman 


Marion Pritchard 


Wallace 


Myma Hines 


Lester R. Rees 


'1 


Sadye Jones 





